{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 第9篇：文本数据处理\n",
    "pandas作为一款强大的数据处理和分析工具，其强大的原因之一就是针对各种常见类型的数据都提供了非常方便的接口，只需要修改指定类型的调用名称，而沿用pandas通用的调用方式，使其具有易学性、易用性、灵活性和可扩展性的特点，这也是为什么pandas如何火爆流行的原因。今天我们学习的就是在工作中最常见的数据类型——文本数据，看看pandas为这种类型的数据提供了哪些方便的功能。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 第一部分：文本数据类型\n",
    "object 和 StringDtype 是 Pandas 的两个文本类型，不过作为新的数据类型，官方推荐 StringDtype 的使用。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "导入相关包"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### object\n",
    "默认情况下，文本数据会被推断为 object 类型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0      Tom\n",
       "1      Bob\n",
       "2     Mary\n",
       "3    James\n",
       "4     Andy\n",
       "5    Alice\n",
       "Name: name, dtype: object"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(data=[\"Tom\", \"Bob\", \"Mary\", \"James\", \"Andy\", \"Alice\"], name=\"name\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### string\n",
    "string 类型需要专门进行指定："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0      Tom\n",
       "1      Bob\n",
       "2     Mary\n",
       "3    James\n",
       "4     Andy\n",
       "5    Alice\n",
       "Name: name, dtype: string"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(data=[\"Tom\", \"Bob\", \"Mary\", \"James\", \"Andy\", \"Alice\"], name=\"name\", dtype='string')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0      Tom\n",
       "1      Bob\n",
       "2     Mary\n",
       "3    James\n",
       "4     Andy\n",
       "5    Alice\n",
       "Name: name, dtype: string"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(data=[\"Tom\", \"Bob\", \"Mary\", \"James\", \"Andy\", \"Alice\"], name=\"name\", dtype=pd.StringDtype())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 类型转换"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0      Tom\n",
       "1      Bob\n",
       "2     Mary\n",
       "3    James\n",
       "4     Andy\n",
       "5    Alice\n",
       "dtype: string"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(data=[\"Tom\", \"Bob\", \"Mary\", \"James\", \"Andy\", \"Alice\"]).astype('string')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0      Tom\n",
       "1      Bob\n",
       "2     Mary\n",
       "3    James\n",
       "4     Andy\n",
       "5    Alice\n",
       "Name: name, dtype: object"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(data=[\"Tom\", \"Bob\", \"Mary\", \"James\", \"Andy\", \"Alice\"], name=\"name\", dtype='str').astype('object')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "#pd.Series([1,'1.']).astype('string') #报错\n",
    "#pd.Series([1,2]).astype('string') #报错\n",
    "#pd.Series([True,False]).astype('string') #报错"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "当下正确的方法是分两部转换，先转为str型object，在转为string类型："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     1\n",
       "1    1.\n",
       "dtype: string"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series([1,'1.']).astype('str').astype('string')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    1\n",
       "1    2\n",
       "dtype: string"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series([1,2]).astype('str').astype('string')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     True\n",
       "1    False\n",
       "dtype: string"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series([True,False]).astype('str').astype('string')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 字符操作\n",
    "Series 和 Index 都用一些字符串处理方法，可以方便地进行操作，这些方法会自动排除丢失值和 NA 值。我们可以通过 str 属性访问它的方法，进行操作。\n",
    "\n",
    "我们可以使用 .str.<method> 访问器（Accessors）来对内容进行字符操作："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在之前已经了解过，在对 Series 中每个元素处理时，我们可以使用 map 或 apply 方法。比如，我想要将每个城市都转为小写，可以使用如下的方式。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0      tom\n",
       "1      bob\n",
       "2     mary\n",
       "3    james\n",
       "4     andy\n",
       "5    alice\n",
       "dtype: object"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(data=[\"Tom\", \"Bob\", \"Mary\", \"James\", \"Andy\", \"Alice\"]).map(lambda x: x.lower())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "str属性可以直接用lower方法实现"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0      tom\n",
       "1      bob\n",
       "2     mary\n",
       "3    james\n",
       "4     andy\n",
       "5    alice\n",
       "dtype: object"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(data=[\"Tom\", \"Bob\", \"Mary\", \"James\", \"Andy\", \"Alice\"]).str.lower()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    3\n",
       "1    3\n",
       "2    4\n",
       "3    5\n",
       "4    4\n",
       "5    5\n",
       "dtype: int64"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(data=[\"Tom\", \"Bob\", \"Mary\", \"James\", \"Andy\", \"Alice\"]).map(lambda x: len(x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    3\n",
       "1    3\n",
       "2    4\n",
       "3    5\n",
       "4    4\n",
       "5    5\n",
       "dtype: int64"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(data=[\"Tom\", \"Bob\", \"Mary\", \"James\", \"Andy\", \"Alice\"]).str.len()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果对数据连续进行字符操作，则每个操作都要使用 .str 方法："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0       tom\n",
       "1     b_o_b\n",
       "2     m_ary\n",
       "3     james\n",
       "4     a_ndy\n",
       "5    a_lice\n",
       "dtype: object"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(data=[\"Tom \", \"B o b\", \"M ary \", \"James \", \"A ndy\", \"A lice\"]).str.strip().str.lower().str.replace(' ', '_')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对于返回布尔值的操作可以用到Series和DataFrame布尔查找中进行逻辑筛选。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    Tom\n",
       "dtype: object"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "name = pd.Series(data=[\"Tom\", \"Bob\", \"Mary\", \"James\", \"Andy\", \"Alice\"])\n",
    "name[name.str.contains('T')]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### object和string对比\n",
    "string类型和object不同之处有三： \n",
    "- ① 字符存取方法（string accessor methods，如str.count）会返回相应数据的Nullable类型，而object会随缺失值的存在而改变返回类型\n",
    "- ② 某些Series方法不能在string上使用，例如： Series.str.decode()，因为存储的是字符串而不是字节\n",
    "- ③ string类型在缺失值存储或运算时，类型会广播为pd.NA，而不是浮点型np.nan\n",
    "\n",
    "其余全部内容在当前版本下完全一致，但迎合Pandas的发展模式，我们仍然全部用string来操作字符串"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    0.0\n",
       "1    0.0\n",
       "2    NaN\n",
       "3    0.0\n",
       "4    1.0\n",
       "5    1.0\n",
       "dtype: float64"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(data=[\"Tom\", \"Bob\", None, \"James\", \"Andy\", \"Alice\"]).str.count('A')  # float64"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0       0\n",
       "1       0\n",
       "2    <NA>\n",
       "3       0\n",
       "4       1\n",
       "5       1\n",
       "Name: name, dtype: Int64"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(data=[\"Tom\", \"Bob\", None, \"James\", \"Andy\", \"Alice\"], name=\"name\", dtype=pd.StringDtype()).str.count('A')  # int64"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    False\n",
       "1    False\n",
       "2     None\n",
       "3    False\n",
       "4    False\n",
       "5    False\n",
       "dtype: object"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(data=[\"Tom\", \"Bob\", None, \"James\", \"Andy\", \"Alice\"]).str.isdigit()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    False\n",
       "1    False\n",
       "2     <NA>\n",
       "3    False\n",
       "4    False\n",
       "5    False\n",
       "Name: name, dtype: boolean"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(data=[\"Tom\", \"Bob\", None, \"James\", \"Andy\", \"Alice\"], name=\"name\", dtype=pd.StringDtype()).str.isdigit()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "类似于 Series.str.decode() 在 StringDtype 上不可用，因为 StringArray 只保存字符串，而不是字节。在比较操作中，基于 StringArray 的 arrays.StringArray 和 Series 将返回一个 BooleanDtype 对象。\n",
    "其余的方法 string 和 object 的操作都相同。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 第二部分：文本处理方法\n",
    "\n",
    "这里列出了一些常用的方法摘要。\n",
    "\n",
    "- cat()   连接字符串\n",
    "- split() 在分隔符上分割字符串\n",
    "- rsplit()    从字符串末尾开始分隔字符串\n",
    "- get()   索引到每个元素（检索第i个元素）\n",
    "- join()  使用分隔符在系列的每个元素中加入字符串\n",
    "- get_dummies()   在分隔符上分割字符串，返回虚拟变量的DataFrame\n",
    "- contains()  如果每个字符串都包含pattern / regex，则返回布尔数组\n",
    "- replace()   用其他字符串替换pattern / regex的出现\n",
    "- repeat()    重复值（s.str.repeat(3)等同于x * 3 t2 >）\n",
    "- pad()   将空格添加到字符串的左侧，右侧或两侧\n",
    "- center()    相当于str.center\n",
    "- ljust() 相当于str.ljust\n",
    "- rjust() 相当于str.rjust\n",
    "- zfill() 等同于str.zfill\n",
    "- wrap()  将长长的字符串拆分为长度小于给定宽度的行\n",
    "- slice() 切分Series中的每个字符串\n",
    "- slice_replace() 用传递的值替换每个字符串中的切片\n",
    "- count() 计数模式的发生\n",
    "- startswith()    相当于每个元素的str.startswith(pat)\n",
    "- endswith()  相当于每个元素的str.endswith(pat)\n",
    "- findall()   计算每个字符串的所有模式/正则表达式的列表\n",
    "- match() 在每个元素上调用re.match，返回匹配的组作为列表\n",
    "- extract()   在每个元素上调用re.search，为每个元素返回一行DataFrame，为每个正则表达式捕获组返回一列\n",
    "- extractall()    在每个元素上调用re.findall，为每个匹配返回一行DataFrame，为每个正则表达式捕获组返回一列\n",
    "- len()   计算字符串长度\n",
    "- strip() 相当于str.strip\n",
    "- rstrip()    相当于str.rstrip\n",
    "- lstrip()    相当于str.lstrip\n",
    "- partition() 等同于str.partition\n",
    "- rpartition()    等同于str.rpartition\n",
    "- lower() 相当于str.lower\n",
    "- upper() 相当于str.upper\n",
    "- find()  相当于str.find\n",
    "- rfind() 相当于str.rfind\n",
    "- index() 相当于str.index\n",
    "- rindex()    相当于str.rindex\n",
    "- capitalize()    相当于str.capitalize\n",
    "- swapcase()  相当于str.swapcase\n",
    "- normalize() 返回Unicode标准格式。相当于unicodedata.normalize\n",
    "- translate() 等同于str.translate\n",
    "- isalnum()   等同于str.isalnum\n",
    "- isalpha()   等同于str.isalpha\n",
    "- isdigit()   相当于str.isdigit\n",
    "- isspace()   等同于str.isspace\n",
    "- islower()   相当于str.islower\n",
    "- isupper()   相当于str.isupper\n",
    "- istitle()   相当于str.istitle\n",
    "- isnumeric() 相当于str.isnumeric\n",
    "- isdecimal() 相当于str.isdecimal"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "数据准备"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>city</th>\n",
       "      <th>hobby</th>\n",
       "      <th>birth</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Tom</td>\n",
       "      <td>Bei Jing</td>\n",
       "      <td>羽毛球;乒乓球</td>\n",
       "      <td>2000-02-10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Bob</td>\n",
       "      <td>Shang Hai</td>\n",
       "      <td>棒球;橄榄球</td>\n",
       "      <td>1988-10-17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Mary</td>\n",
       "      <td>Guang Zhou</td>\n",
       "      <td>舞蹈</td>\n",
       "      <td>2000-01-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>James</td>\n",
       "      <td>Shen Zhen</td>\n",
       "      <td>慈善;做饭</td>\n",
       "      <td>1978-08-08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Yafei</td>\n",
       "      <td>Jin Cheng</td>\n",
       "      <td>篮球;国学;编程</td>\n",
       "      <td>2008-08-08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Alice</td>\n",
       "      <td>Zhen Zhou</td>\n",
       "      <td>跑步;篮球</td>\n",
       "      <td>1988-10-17</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    name        city     hobby       birth\n",
       "0    Tom   Bei Jing    羽毛球;乒乓球  2000-02-10\n",
       "1    Bob  Shang Hai     棒球;橄榄球  1988-10-17\n",
       "2   Mary  Guang Zhou        舞蹈  2000-01-01\n",
       "3  James   Shen Zhen     慈善;做饭  1978-08-08\n",
       "4  Yafei   Jin Cheng  篮球;国学;编程  2008-08-08\n",
       "5  Alice   Zhen Zhou     跑步;篮球  1988-10-17"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = {\n",
    "    \"name\": [\"Tom\", \"Bob\", \"Mary\", \"James\", \"Yafei\", \"Alice\"],\n",
    "    \"city\": [\"Bei Jing \", \"Shang Hai \", \"Guang Zhou\", \"Shen Zhen\", \"Jin Cheng\", \"Zhen Zhou\"],\n",
    "    \"hobby\": ['羽毛球;乒乓球', '棒球;橄榄球', '舞蹈', '慈善;做饭', '篮球;国学;编程', '跑步;篮球'],\n",
    "    \"birth\": [\"2000-02-10\", \"1988-10-17\", \"2000-01-01\", \"1978-08-08\", \"2008-08-08\", \"1988-10-17\"]\n",
    "}\n",
    "user_info = pd.DataFrame(data=data)\n",
    "user_info"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 文本分割\n",
    "对文本的分隔和替换是最常用的文本处理方式。对文本分隔后会生成一个列表，我们对列表进行切片操作，可以找到我们想要的内容，分隔后还可以将分隔内容展开，形成单独的行。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### str.split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    [2000, 02, 10]\n",
       "1    [1988, 10, 17]\n",
       "2    [2000, 01, 01]\n",
       "3    [1978, 08, 08]\n",
       "4    [2008, 08, 08]\n",
       "5    [1988, 10, 17]\n",
       "Name: birth, dtype: object"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_info['birth'].str.split('-')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这里需要注意split后的类型是object，因为现在Series中的元素已经不是string，而包含了list，且string类型只能含有字符串\n",
    "对于str方法可以进行元素的选择，如果该单元格元素是列表，那么str[i]表示取出第i个元素，如果是单个元素，则先把元素转为列表在取出"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    2000\n",
       "1    1988\n",
       "2    2000\n",
       "3    1978\n",
       "4    2008\n",
       "5    1988\n",
       "Name: birth, dtype: object"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_info['birth'].str.split('-').str[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    02\n",
       "1    10\n",
       "2    01\n",
       "3    08\n",
       "4    08\n",
       "5    10\n",
       "Name: birth, dtype: object"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_info['birth'].str.split('-').str.get(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    [02, 10]\n",
       "1    [10, 17]\n",
       "2    [01, 01]\n",
       "3    [08, 08]\n",
       "4    [08, 08]\n",
       "5    [10, 17]\n",
       "Name: birth, dtype: object"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_info['birth'].str.split('-').str[1:3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**字符展开**  \n",
    "expand参数控制了是否将列拆开，n参数代表最多分割多少次"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2000</td>\n",
       "      <td>02</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1988</td>\n",
       "      <td>10</td>\n",
       "      <td>17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2000</td>\n",
       "      <td>01</td>\n",
       "      <td>01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1978</td>\n",
       "      <td>08</td>\n",
       "      <td>08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2008</td>\n",
       "      <td>08</td>\n",
       "      <td>08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>1988</td>\n",
       "      <td>10</td>\n",
       "      <td>17</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      0   1   2\n",
       "0  2000  02  10\n",
       "1  1988  10  17\n",
       "2  2000  01  01\n",
       "3  1978  08  08\n",
       "4  2008  08  08\n",
       "5  1988  10  17"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_info['birth'].str.split('-', expand=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "限制分隔的次数，从左开始，剩余的不分隔"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    [2000, 02-10]\n",
       "1    [1988, 10-17]\n",
       "2    [2000, 01-01]\n",
       "3    [1978, 08-08]\n",
       "4    [2008, 08-08]\n",
       "5    [1988, 10-17]\n",
       "Name: birth, dtype: object"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_info['birth'].str.split('-', n=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2000</td>\n",
       "      <td>02-10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1988</td>\n",
       "      <td>10-17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2000</td>\n",
       "      <td>01-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1978</td>\n",
       "      <td>08-08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2008</td>\n",
       "      <td>08-08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>1988</td>\n",
       "      <td>10-17</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      0      1\n",
       "0  2000  02-10\n",
       "1  1988  10-17\n",
       "2  2000  01-01\n",
       "3  1978  08-08\n",
       "4  2008  08-08\n",
       "5  1988  10-17"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_info['birth'].str.split('-', n=1, expand=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**使用正则**  \n",
    "对于规则比较复杂的，分隔符处可以传入正则表达式："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     [Be, ing ]\n",
       "1    [Shan, ai ]\n",
       "2    [Guan, hou]\n",
       "3     [She, hen]\n",
       "4     [Ji, heng]\n",
       "5     [Zhe, hou]\n",
       "Name: city, dtype: object"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_info.city.str.split(f'[A-z]\\s[A-z]')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### str.rsplit\n",
    "rsplit 和 split一样，只不过它是从右边开始分隔，如果没有n参数，rsplit和split的输出是相同的。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2000-02</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1988-10</td>\n",
       "      <td>17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2000-01</td>\n",
       "      <td>01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1978-08</td>\n",
       "      <td>08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2008-08</td>\n",
       "      <td>08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>1988-10</td>\n",
       "      <td>17</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         0   1\n",
       "0  2000-02  10\n",
       "1  1988-10  17\n",
       "2  2000-01  01\n",
       "3  1978-08  08\n",
       "4  2008-08  08\n",
       "5  1988-10  17"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_info['birth'].str.rsplit('-', n=1, expand=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 文本替换\n",
    "对数据处理时我们可以使用替换功能剔除我们不想要的，换成想要的内容。这在数据处理中经常使用，因为经过人工整理的数据往往不理想，需要进行替换操作。广义上的替换，就是指str.replace函数的应用，fillna是针对缺失值的替换，之前已经提及。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### str.replace"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    2000/02/10\n",
       "1    1988/10/17\n",
       "2    2000/01/01\n",
       "3    1978/08/08\n",
       "4    2008/08/08\n",
       "5    1988/10/17\n",
       "Name: birth, dtype: object"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_info['birth'].str.replace('-', '/')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "第一个值写r开头的正则表达式，后一个写替换的字符串"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0      Bei Jing\n",
       "1     Shang Hai\n",
       "2    Guang Zhou\n",
       "3     Shen Zhen\n",
       "4     Jin Cheng\n",
       "5     Zhen Zhou\n",
       "Name: city, dtype: object"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_info['city'].str.replace('\\s$|^\\s', '', regex=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "使用函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     ieB gniJ \n",
       "1    gnahS iaH \n",
       "2    gnauG uohZ\n",
       "3     nehS nehZ\n",
       "4     niJ gnehC\n",
       "5     nehZ uohZ\n",
       "Name: city, dtype: object"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def func(text):\n",
    "    return text.group(0)[::-1]\n",
    "    \n",
    "user_info['city'].str.replace('[A-z]+', func, regex=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 文本拼接"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### str.cat"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "cat方法对于不同对象的作用结果并不相同，其中的对象包括：单列、双列、多列  \n",
    "① 对于单个Series而言，就是指所有的元素进行字符合并为一个字符串"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'羽毛球;乒乓球棒球;橄榄球舞蹈慈善;做饭篮球;国学;编程跑步;篮球'"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_info.hobby.str.cat()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其中可选sep分隔符参数，和缺失值替代字符na_rep参数,na_rep='*'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'羽毛球;乒乓球;棒球;橄榄球;舞蹈;慈善;做饭;篮球;国学;编程;跑步;篮球'"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_info.hobby.str.cat(sep=';')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "② 对于两个Series合并而言，是对应索引的元素进行合并"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0       羽毛球;乒乓球Tom\n",
       "1        棒球;橄榄球Bob\n",
       "2           舞蹈Mary\n",
       "3       慈善;做饭James\n",
       "4    篮球;国学;编程Yafei\n",
       "5       跑步;篮球Alice\n",
       "Name: hobby, dtype: object"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_info.hobby.str.cat(user_info.name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "同样也有相应参数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "ename": "SyntaxError",
     "evalue": "invalid character in identifier (<ipython-input-41-aa972d0c4536>, line 1)",
     "output_type": "error",
     "traceback": [
      "\u001b[1;36m  File \u001b[1;32m\"<ipython-input-41-aa972d0c4536>\"\u001b[1;36m, line \u001b[1;32m1\u001b[0m\n\u001b[1;33m    ③ 多列拼接\u001b[0m\n\u001b[1;37m    ^\u001b[0m\n\u001b[1;31mSyntaxError\u001b[0m\u001b[1;31m:\u001b[0m invalid character in identifier\n"
     ]
    }
   ],
   "source": [
    "③ 多列拼接"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0       羽毛球;乒乓球TomBei Jing \n",
       "1       棒球;橄榄球BobShang Hai \n",
       "2          舞蹈MaryGuang Zhou\n",
       "3       慈善;做饭JamesShen Zhen\n",
       "4    篮球;国学;编程YafeiJin Cheng\n",
       "5       跑步;篮球AliceZhen Zhou\n",
       "Name: hobby, dtype: object"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_info.hobby.str.cat(user_info[['name', 'city']])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "④对齐方式"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "s = pd.Series(['a', 'b', 'c', 'd'], dtype=\"string\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "u = pd.Series(['b', 'd', 'a', 'c'],\n",
    "              index=[1, 3, 0, 2],\n",
    "              dtype=\"string\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    aa\n",
       "1    bb\n",
       "2    cc\n",
       "3    dd\n",
       "dtype: string"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 以左边索的为主\n",
    "s.str.cat(u)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    aa\n",
       "1    bb\n",
       "2    cc\n",
       "3    dd\n",
       "dtype: string"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.str.cat(u, join='left')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1    bb\n",
       "3    dd\n",
       "0    aa\n",
       "2    cc\n",
       "dtype: string"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 以右边的索引为主\n",
    "s.str.cat(u, join='right')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    aa\n",
       "1    bb\n",
       "2    c-\n",
       "3    dd\n",
       "dtype: string"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 其他\n",
    "t = pd.Series(['a', 'b', np.nan, 'd'], dtype=\"string\")\n",
    "s.str.cat(t, join='outer', na_rep='-')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    aa\n",
       "1    bb\n",
       "2    c-\n",
       "3    dd\n",
       "dtype: string"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.str.cat(t, join='inner', na_rep='-')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 文本匹配与提取\n",
    "既然是在操作字符串，很自然，你可能会想到是否可以从一个长的字符串中提取出子串。答案是可以的。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### str.extract"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 提取第一个匹配的子串\n",
    "　　extract 方法接受一个正则表达式并至少包含一个捕获组，指定参数 expand=True 可以保证每次都返回 DataFrame。\n",
    "　　例如，现在想要匹配空字符串前面的所有的字母，可以使用如下操作：如果使用多个组提取正则表达式会返回一个 DataFrame，每个组只有一列。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Bei</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Shang</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Guang</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Shen</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Jin</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Zhen</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       0\n",
       "0    Bei\n",
       "1  Shang\n",
       "2  Guang\n",
       "3   Shen\n",
       "4    Jin\n",
       "5   Zhen"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_info.city.str.extract(\"(\\w+)\\s+\", expand=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "想要匹配出空字符串前面和后面的所有字母，操作如下"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Bei</td>\n",
       "      <td>Jing</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Shang</td>\n",
       "      <td>Hai</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Guang</td>\n",
       "      <td>Zhou</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Shen</td>\n",
       "      <td>Zhen</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Jin</td>\n",
       "      <td>Cheng</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Zhen</td>\n",
       "      <td>Zhou</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       0      1\n",
       "0    Bei   Jing\n",
       "1  Shang    Hai\n",
       "2  Guang   Zhou\n",
       "3   Shen   Zhen\n",
       "4    Jin  Cheng\n",
       "5   Zhen   Zhou"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_info.city.str.extract(\"(\\w+)\\s+(\\w+)\", expand=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### str.extractall\n",
    "##### 匹配所有子串\n",
    "extract 只能够匹配出第一个子串，使用 extractall 可以匹配出所有的子串。\n",
    "例如，将所有组的空白字符串前面的字母都匹配出来，可以如下操作。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>match</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">0</th>\n",
       "      <th>0</th>\n",
       "      <td>Bei</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Jing</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">1</th>\n",
       "      <th>0</th>\n",
       "      <td>Shang</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Hai</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <th>0</th>\n",
       "      <td>Guang</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <th>0</th>\n",
       "      <td>Shen</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <th>0</th>\n",
       "      <td>Jin</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <th>0</th>\n",
       "      <td>Zhen</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             0\n",
       "  match       \n",
       "0 0        Bei\n",
       "  1       Jing\n",
       "1 0      Shang\n",
       "  1        Hai\n",
       "2 0      Guang\n",
       "3 0       Shen\n",
       "4 0        Jin\n",
       "5 0       Zhen"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_info.city.str.extractall(\"(\\w+)\\s+\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### str.contains"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 测试是否包含子串\n",
    "除了可以匹配出子串外，我们还可以使用 contains 来测试是否包含子串。例如，想要测试城市是否包含子串 “Zh”。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    False\n",
       "1    False\n",
       "2     True\n",
       "3     True\n",
       "4    False\n",
       "5     True\n",
       "Name: city, dtype: bool"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_info.city.str.contains(\"Zh\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     True\n",
       "1    False\n",
       "2     True\n",
       "3    False\n",
       "4    False\n",
       "5    False\n",
       "Name: birth, dtype: bool"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_info.birth.str.contains(r\"^2000\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### str.get_dummies\n",
    "可以从字符串列中提取虚拟变量。 例如用“ |”分隔："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Bei</th>\n",
       "      <th>Cheng</th>\n",
       "      <th>Guang</th>\n",
       "      <th>Hai</th>\n",
       "      <th>Jin</th>\n",
       "      <th>Jing</th>\n",
       "      <th>Shang</th>\n",
       "      <th>Shen</th>\n",
       "      <th>Zhen</th>\n",
       "      <th>Zhou</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Bei  Cheng  Guang  Hai  Jin  Jing  Shang  Shen  Zhen  Zhou\n",
       "0    1      0      0    0    0     1      0     0     0     0\n",
       "1    0      0      0    1    0     0      1     0     0     0\n",
       "2    0      0      1    0    0     0      0     0     0     1\n",
       "3    0      0      0    0    0     0      0     1     1     0\n",
       "4    0      1      0    0    1     0      0     0     0     0\n",
       "5    0      0      0    0    0     0      0     0     1     1"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_info.city.str.get_dummies(sep=\" \")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 其他常用方法"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "文本格式\n",
    "s = pd.Series(['lower', 'CAPITALS', 'this is a sentence', 'SwApCaSe'])\n",
    "s.str.lower() # 转为小写\n",
    "s.str.upper() # 转为大写\n",
    "s.str.title() # 标题格式，每个单词大写\n",
    "s.str.capitalize() # 首字母大写\n",
    "s.str.swapcase() # 大小写互换\n",
    "s.str.casefold() # 转为小写，支持其他语言如德语\n",
    "对齐\n",
    "# 居中对齐，宽度为10，用 - 填充\n",
    "s.str.center(10, fillchar='-')\n",
    "# 左对齐\n",
    "s.str.ljust(10, fillchar='-')\n",
    "# 右对齐\n",
    "s.str.rjust(10, fillchar='-')\n",
    "# 指定宽度，填充内容对齐方式，填充内容\n",
    "# side{‘left’, ‘right’, ‘both’}, default ‘left’\n",
    "s.str.pad(width=10, side='left', fillchar='-')\n",
    "# 填充对齐\n",
    "s.str.zfill(3) # 生成字符，不足3位的前边加0\n",
    "计算\n",
    "# 指定字母的数量\n",
    "s.str.count('a')\n",
    "# 支持正则，包含 abc 三个字母的总数\n",
    "s.str.count(r'a|b|c')\n",
    "# 字符长度\n",
    "s.str.len()\n",
    "编码\n",
    "# 编码\n",
    "s.str.encode('utf-8')\n",
    "# 解码\n",
    "s.str.decode('utf-8')\n",
    "# 字符串的Unicode普通格式\n",
    "# form{‘NFC’, ‘NFKC’, ‘NFD’, ‘NFKD’}\n",
    "s.str.normalize('NFC')\n",
    "类别判断\n",
    "s.str.isalpha # 所有的是否字母\n",
    "s.str.isnumeric # 所有是否数字 0-9\n",
    "s.str.isalnum # 所有的是否字母数字\n",
    "s.str.isdigit # 所有是否数字\n",
    "s.str.isdecimal # 所有是否小数\n",
    "s.str.isspace # 所有是否空格\n",
    "s.str.islower # 所有是否小写\n",
    "s.str.isupper # 所有是否大写\n",
    "s.str.istitle # 所有是否标题格式\n",
    "其他\n",
    "# 类似 python str.translate() 翻译映射\n",
    "s.str.translate(trantab)\n",
    "# 将长文本拆分开指定宽度的字符，用换行连接\n",
    "s.str.wrap(10)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 第三部分：文本处理案例\n",
    "\n",
    "数据集是黄同学精心为大家编造，只为了帮助大家学习到知识。数据集如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>姓名</th>\n",
       "      <th>英文名</th>\n",
       "      <th>性别</th>\n",
       "      <th>身份证</th>\n",
       "      <th>身高</th>\n",
       "      <th>家庭住址</th>\n",
       "      <th>电话号码</th>\n",
       "      <th>收入</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>黄同学</td>\n",
       "      <td>Huang tong_xue</td>\n",
       "      <td>男</td>\n",
       "      <td>463895200003128433</td>\n",
       "      <td>mid:175_good</td>\n",
       "      <td>湖北广水</td>\n",
       "      <td>13434813546</td>\n",
       "      <td>1.1万</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>黄至尊</td>\n",
       "      <td>huang zhi_zun</td>\n",
       "      <td>women</td>\n",
       "      <td>429475199912122345</td>\n",
       "      <td>low:165_bad</td>\n",
       "      <td>河南信阳</td>\n",
       "      <td>19748672895</td>\n",
       "      <td>8.5千</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>黄老邪</td>\n",
       "      <td>Huang Lao_xie</td>\n",
       "      <td>men</td>\n",
       "      <td>420934199110102311</td>\n",
       "      <td>low:159_bad</td>\n",
       "      <td>广西桂林</td>\n",
       "      <td>16728613064</td>\n",
       "      <td>0.9万</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>陈大美</td>\n",
       "      <td>Chen Da_mei</td>\n",
       "      <td>女</td>\n",
       "      <td>431085200005230122</td>\n",
       "      <td>high:180_verygood</td>\n",
       "      <td>湖北孝感</td>\n",
       "      <td>14561586431</td>\n",
       "      <td>6.5千</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>孙尚香</td>\n",
       "      <td>sun shang_xiang</td>\n",
       "      <td>男</td>\n",
       "      <td>420953199509082345</td>\n",
       "      <td>low:172_bad</td>\n",
       "      <td>广东广州</td>\n",
       "      <td>19384683910</td>\n",
       "      <td>2.0万</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     姓名              英文名     性别                 身份证                 身高  家庭住址  \\\n",
       "0   黄同学   Huang tong_xue      男  463895200003128433       mid:175_good  湖北广水   \n",
       "1   黄至尊    huang zhi_zun  women  429475199912122345        low:165_bad  河南信阳   \n",
       "2  黄老邪     Huang Lao_xie    men  420934199110102311        low:159_bad  广西桂林   \n",
       "3   陈大美      Chen Da_mei      女  431085200005230122  high:180_verygood  湖北孝感   \n",
       "4   孙尚香  sun shang_xiang      男  420953199509082345        low:172_bad  广东广州   \n",
       "\n",
       "          电话号码    收入  \n",
       "0  13434813546  1.1万  \n",
       "1  19748672895  8.5千  \n",
       "2  16728613064  0.9万  \n",
       "3  14561586431  6.5千  \n",
       "4  19384683910  2.0万  "
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df ={'姓名':[' 黄同学','黄至尊','黄老邪 ','陈大美','孙尚香'],\n",
    "     '英文名':['Huang tong_xue','huang zhi_zun','Huang Lao_xie','Chen Da_mei','sun shang_xiang'],\n",
    "     '性别':['男','women','men','女','男'],\n",
    "     '身份证':['463895200003128433','429475199912122345','420934199110102311','431085200005230122','420953199509082345'],\n",
    "     '身高':['mid:175_good','low:165_bad','low:159_bad','high:180_verygood','low:172_bad'],\n",
    "     '家庭住址':['湖北广水','河南信阳','广西桂林','湖北孝感','广东广州'],\n",
    "     '电话号码':['13434813546','19748672895','16728613064','14561586431','19384683910'],\n",
    "     '收入':['1.1万','8.5千','0.9万','6.5千','2.0万']}\n",
    "df = pd.DataFrame(df)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "观察上述数据，数据集是乱的。接下来，我们就用16个Pandas来对上述数据，进行数据清洗。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ① cat函数：用于字符串的拼接"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     黄同学---湖北广水\n",
       "1     黄至尊---河南信阳\n",
       "2    黄老邪 ---广西桂林\n",
       "3     陈大美---湖北孝感\n",
       "4     孙尚香---广东广州\n",
       "Name: 姓名, dtype: object"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"姓名\"].str.cat(df[\"家庭住址\"],sep='-'*3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ② contains：判断某个字符串是否包含给定字符"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     True\n",
       "1    False\n",
       "2     True\n",
       "3    False\n",
       "4     True\n",
       "Name: 家庭住址, dtype: bool"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"家庭住址\"].str.contains(\"广\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ③ startswith/endswith：判断某个字符串是否以…开头/结尾"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    False\n",
       "1     True\n",
       "2     True\n",
       "3    False\n",
       "4    False\n",
       "Name: 姓名, dtype: bool"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 第一个行的“ 黄伟”是以空格开头的\n",
    "df[\"姓名\"].str.startswith(\"黄\") "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     True\n",
       "1    False\n",
       "2     True\n",
       "3    False\n",
       "4    False\n",
       "Name: 英文名, dtype: bool"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"英文名\"].str.endswith(\"e\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ④ count：计算给定字符在字符串中出现的次数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    3\n",
       "1    0\n",
       "2    1\n",
       "3    1\n",
       "4    2\n",
       "Name: 电话号码, dtype: int64"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"电话号码\"].str.count(\"3\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ⑤ get：获取指定位置的字符串"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    学\n",
       "1    尊\n",
       "2     \n",
       "3    美\n",
       "4    香\n",
       "Name: 姓名, dtype: object"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"姓名\"].str.get(-1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0         [mid, 175_good]\n",
       "1          [low, 165_bad]\n",
       "2          [low, 159_bad]\n",
       "3    [high, 180_verygood]\n",
       "4          [low, 172_bad]\n",
       "Name: 身高, dtype: object"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"身高\"].str.split(\":\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     mid\n",
       "1     low\n",
       "2     low\n",
       "3    high\n",
       "4     low\n",
       "Name: 身高, dtype: object"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"身高\"].str.split(\":\").str.get(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ⑥ len：计算字符串长度"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    1\n",
       "1    5\n",
       "2    3\n",
       "3    1\n",
       "4    1\n",
       "Name: 性别, dtype: int64"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"性别\"].str.len()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ⑦ upper/lower：英文大小写转换"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     HUANG TONG_XUE\n",
       "1      HUANG ZHI_ZUN\n",
       "2      HUANG LAO_XIE\n",
       "3        CHEN DA_MEI\n",
       "4    SUN SHANG_XIANG\n",
       "Name: 英文名, dtype: object"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"英文名\"].str.upper()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     huang tong_xue\n",
       "1      huang zhi_zun\n",
       "2      huang lao_xie\n",
       "3        chen da_mei\n",
       "4    sun shang_xiang\n",
       "Name: 英文名, dtype: object"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"英文名\"].str.lower()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ⑧ pad+side参数/center：在字符串的左边、右边或左右两边添加给定字符"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    ******湖北广水\n",
       "1    ******河南信阳\n",
       "2    ******广西桂林\n",
       "3    ******湖北孝感\n",
       "4    ******广东广州\n",
       "Name: 家庭住址, dtype: object"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"家庭住址\"].str.pad(10,fillchar=\"*\")      # 相当于ljust()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    湖北广水******\n",
       "1    河南信阳******\n",
       "2    广西桂林******\n",
       "3    湖北孝感******\n",
       "4    广东广州******\n",
       "Name: 家庭住址, dtype: object"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"家庭住址\"].str.pad(10,side=\"right\",fillchar=\"*\")    # 相当于rjust()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    ***湖北广水***\n",
       "1    ***河南信阳***\n",
       "2    ***广西桂林***\n",
       "3    ***湖北孝感***\n",
       "4    ***广东广州***\n",
       "Name: 家庭住址, dtype: object"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"家庭住址\"].str.center(10,fillchar=\"*\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ⑨ repeat：重复字符串几次"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0                男男男\n",
       "1    womenwomenwomen\n",
       "2          menmenmen\n",
       "3                女女女\n",
       "4                男男男\n",
       "Name: 性别, dtype: object"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"性别\"].str.repeat(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ⑩ slice_replace：使用给定的字符串，替换指定的位置的字符"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    1343****546\n",
       "1    1974****895\n",
       "2    1672****064\n",
       "3    1456****431\n",
       "4    1938****910\n",
       "Name: 电话号码, dtype: object"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"电话号码\"].str.slice_replace(4,8,\"*\"*4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ⑪ replace：将指定位置的字符，替换为给定的字符串"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0         mid-175_good\n",
       "1          low-165_bad\n",
       "2          low-159_bad\n",
       "3    high-180_verygood\n",
       "4          low-172_bad\n",
       "Name: 身高, dtype: object"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"身高\"].str.replace(\":\",\"-\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ⑫ replace：将指定位置的字符，替换为给定的字符串(接受正则表达式)\n",
    "replace中传入正则表达式，才叫好用；  \n",
    "先不要管下面这个案例有没有用，你只需要知道，使用正则做数据清洗多好用；"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    正则万\n",
       "1    正则千\n",
       "2    正则万\n",
       "3    正则千\n",
       "4    正则万\n",
       "Name: 收入, dtype: object"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"收入\"].str.replace(\"\\d+\\.\\d+\",\"正则\", regex=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ⑬ split方法+expand参数：搭配join方法功能很强大"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0         [mid, 175_good]\n",
       "1          [low, 165_bad]\n",
       "2          [low, 159_bad]\n",
       "3    [high, 180_verygood]\n",
       "4          [low, 172_bad]\n",
       "Name: 身高, dtype: object"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 普通用法\n",
    "df[\"身高\"].str.split(\":\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>姓名</th>\n",
       "      <th>英文名</th>\n",
       "      <th>性别</th>\n",
       "      <th>身份证</th>\n",
       "      <th>身高</th>\n",
       "      <th>家庭住址</th>\n",
       "      <th>电话号码</th>\n",
       "      <th>收入</th>\n",
       "      <th>身高描述</th>\n",
       "      <th>final身高</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>黄同学</td>\n",
       "      <td>Huang tong_xue</td>\n",
       "      <td>男</td>\n",
       "      <td>463895200003128433</td>\n",
       "      <td>mid:175_good</td>\n",
       "      <td>湖北广水</td>\n",
       "      <td>13434813546</td>\n",
       "      <td>1.1万</td>\n",
       "      <td>mid</td>\n",
       "      <td>175_good</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>黄至尊</td>\n",
       "      <td>huang zhi_zun</td>\n",
       "      <td>women</td>\n",
       "      <td>429475199912122345</td>\n",
       "      <td>low:165_bad</td>\n",
       "      <td>河南信阳</td>\n",
       "      <td>19748672895</td>\n",
       "      <td>8.5千</td>\n",
       "      <td>low</td>\n",
       "      <td>165_bad</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>黄老邪</td>\n",
       "      <td>Huang Lao_xie</td>\n",
       "      <td>men</td>\n",
       "      <td>420934199110102311</td>\n",
       "      <td>low:159_bad</td>\n",
       "      <td>广西桂林</td>\n",
       "      <td>16728613064</td>\n",
       "      <td>0.9万</td>\n",
       "      <td>low</td>\n",
       "      <td>159_bad</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>陈大美</td>\n",
       "      <td>Chen Da_mei</td>\n",
       "      <td>女</td>\n",
       "      <td>431085200005230122</td>\n",
       "      <td>high:180_verygood</td>\n",
       "      <td>湖北孝感</td>\n",
       "      <td>14561586431</td>\n",
       "      <td>6.5千</td>\n",
       "      <td>high</td>\n",
       "      <td>180_verygood</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>孙尚香</td>\n",
       "      <td>sun shang_xiang</td>\n",
       "      <td>男</td>\n",
       "      <td>420953199509082345</td>\n",
       "      <td>low:172_bad</td>\n",
       "      <td>广东广州</td>\n",
       "      <td>19384683910</td>\n",
       "      <td>2.0万</td>\n",
       "      <td>low</td>\n",
       "      <td>172_bad</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     姓名              英文名     性别                 身份证                 身高  家庭住址  \\\n",
       "0   黄同学   Huang tong_xue      男  463895200003128433       mid:175_good  湖北广水   \n",
       "1   黄至尊    huang zhi_zun  women  429475199912122345        low:165_bad  河南信阳   \n",
       "2  黄老邪     Huang Lao_xie    men  420934199110102311        low:159_bad  广西桂林   \n",
       "3   陈大美      Chen Da_mei      女  431085200005230122  high:180_verygood  湖北孝感   \n",
       "4   孙尚香  sun shang_xiang      男  420953199509082345        low:172_bad  广东广州   \n",
       "\n",
       "          电话号码    收入  身高描述       final身高  \n",
       "0  13434813546  1.1万   mid      175_good  \n",
       "1  19748672895  8.5千   low       165_bad  \n",
       "2  16728613064  0.9万   low       159_bad  \n",
       "3  14561586431  6.5千  high  180_verygood  \n",
       "4  19384683910  2.0万   low       172_bad  "
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# split方法，搭配expand参数\n",
    "df[[\"身高描述\",\"final身高\"]] = df[\"身高\"].str.split(\":\",expand=True)\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0         mid?????175_good\n",
       "1          low?????165_bad\n",
       "2          low?????159_bad\n",
       "3    high?????180_verygood\n",
       "4          low?????172_bad\n",
       "Name: 身高, dtype: object"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# split方法搭配join方法\n",
    "df[\"身高\"].str.split(\":\").str.join(\"?\"*5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ⑭ strip/rstrip/lstrip：去除空白符、换行符"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    4\n",
       "1    3\n",
       "2    4\n",
       "3    3\n",
       "4    3\n",
       "Name: 姓名, dtype: int64"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"姓名\"].str.len()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    3\n",
       "1    3\n",
       "2    3\n",
       "3    3\n",
       "4    3\n",
       "Name: 姓名, dtype: int64"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"姓名\"] = df[\"姓名\"].str.strip()\n",
    "df[\"姓名\"].str.len()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ⑮ findall：利用正则表达式，去字符串中匹配，返回查找结果的列表\n",
    "findall使用正则表达式，做数据清洗，真的很香！"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0         mid:175_good\n",
       "1          low:165_bad\n",
       "2          low:159_bad\n",
       "3    high:180_verygood\n",
       "4          low:172_bad\n",
       "Name: 身高, dtype: object"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"身高\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0         [mid, good]\n",
       "1          [low, bad]\n",
       "2          [low, bad]\n",
       "3    [high, verygood]\n",
       "4          [low, bad]\n",
       "Name: 身高, dtype: object"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"身高\"].str.findall(\"[a-zA-Z]+\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ⑯ extract/extractall：接受正则表达式，抽取匹配的字符串(一定要加上括号)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>mid</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>low</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>low</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>high</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>low</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      0\n",
       "0   mid\n",
       "1   low\n",
       "2   low\n",
       "3  high\n",
       "4   low"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"身高\"].str.extract(\"([a-zA-Z]+)\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>match</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">0</th>\n",
       "      <th>0</th>\n",
       "      <td>mid</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>good</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">1</th>\n",
       "      <th>0</th>\n",
       "      <td>low</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>bad</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">2</th>\n",
       "      <th>0</th>\n",
       "      <td>low</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>bad</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">3</th>\n",
       "      <th>0</th>\n",
       "      <td>high</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>verygood</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">4</th>\n",
       "      <th>0</th>\n",
       "      <td>low</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>bad</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                0\n",
       "  match          \n",
       "0 0           mid\n",
       "  1          good\n",
       "1 0           low\n",
       "  1           bad\n",
       "2 0           low\n",
       "  1           bad\n",
       "3 0          high\n",
       "  1      verygood\n",
       "4 0           low\n",
       "  1           bad"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# extractall提取得到复合索引\n",
    "df[\"身高\"].str.extractall(\"([a-zA-Z]+)\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>mid</td>\n",
       "      <td>good</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>low</td>\n",
       "      <td>bad</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>low</td>\n",
       "      <td>bad</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>high</td>\n",
       "      <td>verygood</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>low</td>\n",
       "      <td>bad</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      0         1\n",
       "0   mid      good\n",
       "1   low       bad\n",
       "2   low       bad\n",
       "3  high  verygood\n",
       "4   low       bad"
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# extract搭配expand参数\n",
    "df[\"身高\"].str.extract(\"([a-zA-Z]+).*?([a-zA-Z]+)\",expand=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "好了，今天的文章就到这里，我是张亚飞，一个山沟沟里的小人物。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
